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INFORMATION  CLUSTERING  FOR  BETTER  DECISION-MAKING 


INTRODUCTION 


This  report  makes  four  points  relative  to  the  development  of  interfaces  for  decision  aids  to 
be  used  by  experts.  The  first  three  points  are  background;  the  fourth  is  the  focus  of  this  study, 

1.  To  solve  complex  and  ill-structured  problems,  experts  use  chunks  of  information, 
analogous  to  memory  chunks. 

2.  Decision-making  is  improved  by  better  organization  of  the  information.  (Better 
organized  is  defined  in  terms  of  the  user’s  task.) 

3.  Increasingly,  complex  tasks  are  supported  by  computerized  aids.  The  development  of 
these  aids  is  facilitated  by  a  functional  partitioning  of  the  tasks  for  efficient  engineering  of  the 
algorithms  and  programs. 

4.  In  contrast  to  the  functional  partitioning  to  meet  development  needs,  decision  aids  for 
experts  may  be  improved  by  information  chunking  (clustering)  that  is  consistent  with  the  structure 
of  the  user's  problem-solving  schema,  i.e.,  the  chunks  referred  to  above.  This  last  point  is  the 
kernel  of  an  information-clustering  hypothesis  that  will  be  developed  here. 

The  context  for  this  report  is  the  development  of  computerized  decision  aids  for  higher- 
level  technical  decision-makers.  Although  originally  only  used  by  scientists  and  engineers, 
computers  are  prevalent  everywhere  today  as  word  processors,  data  manipulators  (via 
spreadsheets  and  databases),  scheduling  aids,  and  cybernetic  sensors  and  control  devices. 
Computers  sit  on  the  desks  of  most  clerical,  sales,  financial,  technical,  and  managerial  personnel, 
up  through  middle  management. 

Computers  as  decision  aids  are  now  reaching  into  the  offices  and  control  rooms  of  senior 
decision-makers  and  managers.  For  example,  the  NASA  flight  control  room  we  have  all  seen  on 
television  is  filled  with  individuals  using  computers  to  monitor  and  control  elements  of  the  launch 
process.  However,  the  test  director  has  only  a  television  screen,  a  notebook  of  schedules  and 
protocols,  and  telephones  to  aid  in  keeping  the  process  coordinated  and  on  time  (John, 
Remington,  and  Streier,  1991).  The  most  important  aspect  of  the  test  director’s  job  is  making 
decisions  when  unexpected  events  and  anomalies  arise.  The  need  for,  and  appropriate  design  of, 
dedicated  computer  support  is  just  now  being  investigated.  Because  the  test  director’s  job  is 
more  than  the  sum  of  the  individual  functions,  this  computer  support  must  not  only  allow  the  test 
director  to  look  over  the  shoulders  of  all  the  individuals  controlling  specific  functions,  it  must 
support  the  integrated  evaluation  of  current  status,  schedule  projections,  resources,  and 
limitations  in  support  of  efficient  scheduling  and  planning  for  an  on-time  and  safe  launch.  Most 
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importantly,  computer  support  must  not  impose  constraints  that  will  inhibit  the  solution  of  unique 
problems.  As  with  support  for  the  test  director,  the  current  study  focuses  on  those  domains  for 
which  the  first  generation  of  computer  support  is  being  conceived. 


EXPERT  PROBLEM-SOLVING 

The  literature  on  expert  problem-solving  is  too  lengthy  to  be  reviewed  here.  The  reader  is 
referred  to  Chi,  Glaser,  and  Farr  (1981)  or  Ericsson  and  Smith  (1991)  for  a  more  complete  review 
of  the  topic.  One  point  will  be  discussed  here. 

Part  of  the  elSiciency  of  expertise  can  be  attributed  to  the  complied  nature  of  expert 
knowledge.  Experts  systematically  use  chunks  comprised  of  related  items  of  information  during 
the  situation  assessment  phase  of  decision-making  and  problem-solving.  These  chunks  can  be 
identified  when  experts  search  for  information  in  a  problem-solving,  decision-making,  or  memory 
task  (Kirschenbaum,  1992;  Chase  and  Simon,  1973).  The  chunking  of  information  in  long-term 
memory  is  similar  (and  related)  to  the  chunking  effect  that  improves  capacity  in  short-term 
memory  (Miller,  1956).  Evidence  that  expert  knowledge  is  comprised  of  chunks  of  information 
was  first  reported  by  deOroot  (1965)  in  a  study  of  chess  masters.  In  later  studies  of  expert  recall 
of  chess  positions.  Chase  and  Simon  (1973)  report  that  chess  masters  typically  examine  and 
record  patterns  of  positions  quickly,  with  longer  latencies  between  pattern  groups.  Kirschenbaum 
(1992)  reported  a  similar  temporal  indication  of  the  use  of  chunks  of  information  when 
experienced  submarine  officers  examined  situational  data  prior  to  making  maneuver  decisions. 
These  chunks  correspond  to  aggregated  feature  components  employed  by  operators  working  at 
Rasmussen's  (1983)  skill-based  and  rule-based  levels  of  performance.  Such  aggregated  features 

speed  situation  recognition  and  provide  access  to  preprogrammed  responses.  Mo^er  way  to — . 

think  of  these  chunks  is  as  complied  knowledge.  Both  Anderson's  (1983)  ACT’^  'and  Laird, 
Rosenbloom,  and  Newell's  (1986)  SOAR  formalisms  make  use  of  this  concept  to  explain  how 
experts  perform  so  effortlessly  and  quickly,  within  their  area  of  expertise. 

In  the  past,  information  displays  and  analysis  tools  have  comprised  the  bulk  of  computerized 
decision  aids.  These  have  been  employed  by  subordinate  operators  and  analysts  to  perform 
single-goal  (or  serial-goal)  tasks.  Note  that  hierarchical  decision  teams  are  often  composed  of 
two  (or  more)  levels  of  personnel.  The  lower-level  positions  are  limited  in  their  scope  and 
authority  and  perform  relatively  limited  and  routine  tasks  directed  by  higher  auth  irity. 

Technicians,  nurses,  and  sailors  are  examples.  A  second  hierarchy,  usually  diffi  entiated  by 
educational  credentials,  can  lead  to  senior  executive  level  decision  authority.  Frequently,  even  the 
least  experienced  member  of  this  group  is,  at  least  formally,  superior  to  all  members  of  the  lower 
group.  Examples  of  these  professions  are  scientists,  physicians,  and  military  officers.  In  each 
group  there  is  a  hierarchy  of  experience,  and  movement  between  groups  is  rare.  The  focus  of  this 
investigation  is  the  senior  expert  in  the  second  category  who  is  doing,  rather  than  managing,  the 
task. 


2 


The  task  of  the  senior  decision-maker  (submarine  commanding  officer,  surgeon,  or  NASA 
test  director)  is  to  evaluate  (integrate,  compare  conditions  to  assumptions,  assess,  etc.)  the 
available  information,  predict  the  effects  of  various  action  options,  and  communicate  the  decision. 
The  available  information  is  composed  partially  of  data  previously  analyzed  by  subordinates  and 
partially  of  unanalyzed  raw  data.  It  includes  history,  current  state,  predictions,  and  may  contain 
considerable  uncertainty,  or  may  change  over  time.  Required  decisions  may  include  both  what  to 
do  and  when  to  act. 


INFORMATION  ORGANIZATION 

Recent  studies  (Kleinmuntz  and  Schkade,  1993)  show  that  how  a  problem  is  represented 
affects  the  speed  and  accuracy  of  identifying  and  assessing  the  situation,  and  consequently,  the 
quality  of  the  decisions  made  with  that  information.  For  example,  Johnson,  Payne,  and  Bettman 
(1988)  found  that  display  format  affects  the  likelihood  of  preference  reversals  (a  well-documented 
decision  error)  in  choice  decision-making.  Decision-makers  in  these  studies  shifted  information¬ 
gathering  strategies  as  a  function  of  display  format.  Brown  and  Klayman  (1989)  and  Smith 
(1989)  found  that  representation  affects  subjects'  ability  to  identify  key  problem  elements  in 
naturalistic  decision  situations.  Larkin  (1989)  has  called  this  effect  display-based  problem-solving 
because  the  availability  and  form  of  the  information  displayed  can  affect  problem-solving.  For 
example,  Russo  (1977)  found  that  a  table  of  unit  prices  for  an  entire  category  of  food-facilitated 
price  comparison  and  decision-making,  as  compared  to  unit  prices  displayed  with  each  item, 
although  unit  prices  are  calculated  by  item,  not  category.  One  reason  for  this  improvement  may 
be  the  reduction  in  working  memory  load  when  appropriate  information  is  clustered. 

Note  that  better  organized  ii  defined  in  terms  of  the  user's  task,  not  the  engineering 
components  or  subsystems.  Thus,  better  organization  is  not  necessarily  the  same  as  the  problem 
partitioning  employed  for  data  analysis  or  engineering  development.  In  contrast  to  this  user- 
centered  better  organization,  the  term  Junctional  partitioning  will  be  used  to  refer  to  information 
that  is  partitioned  according  to  engineering  function. 


FUNCTIONAL  PARTTHONING 

The  world  is  becoming  an  increasingly  computerized  place.  Computerized  systems  are 
replacing  traditional  information  source.s  in  complex  environments  like  emergency  rooms,  military 
operation  centers,  and  power  plrnt  control  rooms,  as  well  as  more  mundane  offices  and 
institutions.  To  support  these  activities,  there  has  been  extensive  expertise  applied  to  the 
engineering  and  mathematics  necessary  to  analyze  and  reduce  massive  amounts  of  real-time  data. 
The  engineering  approach  is  to  break  systems  down  into  their  functional  components  to  better 
facilitLte  analysis  and  subsystem  development.  This  has  resulted  in  new  ways  to  reduce,  partition, 
and  analyze  data.  However,  the  engineering  (functional)  partitioning  of  the  components  has  often 
been  carried  over  to  the  design  of  human  teams  to  perform  the  various  tasks.  For  example,  on  the 
modem  Navy  ship,  the  team  organization  parallels  the  engineering  partitioning  of  the  control 
system.  Initially,  the  human  tasks  motivated  the  development  of  computerized  support  systems, 
but  today  the  support  systems  may  dictate  the  tasks  of  the  humans. 
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INFORMATION  CLUSTERING 


Interface  design  has  followed  the  analytic  engineering  development  because  the  developers 
(often  wrongly)  assumed  that  the  users  would  partition  the  task  in  a  like  manner.  Even  when  this 
has  been  true  for  single  task  operators,  it  has  never  been  true  for  high-level  decision  makers. 
Today,  even  the  operator  performing  a  single  task  may  need  to  evaluate  performance  in  terms  of 
other  tasks,  other  variables,  or  the  behavior  of  other  operators.  Thus,  the  engineering  analysis 
that  is  appropriate  to  equipment  may  not  be  consistent  with  user  needs.  Even  with  modem 
multitasking  and  multiple  windows,  functional  partitioning  according  to  engineering  tasks  may 
hinder  efficient  cognitive  information  fUsion  and  integration  required  in  higher-level  human 
problem-solving  (or  perception).  There  is  a  diversity  of  information  that  may  comprise  elements 
in  a  single  chunk.  Herein  the  term  integrate  will  refer  to  bringing  together  these  diverse  elements 
in  a  single  picture.  Integrating  information  may  affect  interpretation,  in  the  same  way  that  an 
interaction  affects  the  interpretation  of  a  statistical  result,  i.e.,  it  is  more  than  the  sum  of  its  parts. 
Bringing  together  the  elements  of  a  chunk  allows  the  expert  to  perceive  and  quickly  recognize  a 
known  situation  in  order  to  respond  more  efficiently  and  accurately.  Relieving  the  expert  of  the 
need  to  integrate  the  elements  of  a  chunk  in  (huntan)  memory  should  reduce  errors  in  chunk 
identification  and  speed  decision-making. 

Thus,  decision  aids  for  experts  performing  integrative  decision  tasks  may  be  improved  by 
employing  an  organizational  schema  for  the  information  that  is  consistent  with  the  deeply 
structured  problem  solving  schema  of  the  human  user.  Decision  support  and  information 
management  systems  have  been  seen  as  both  the  solution  to  the  real-time,  information 
management  problem,  and  its  cause.  That  is,  the  additional  information  that  is  collected  with 
advanced  technology  can  be  presented  electronicaliy  to  the  decision  makers.  Even  had  it  been 
available,  much  of  the  additional  information  would  have  been  lost  in  the  days  of  only  pencils  and 
paper.  However,  the  complaint  today  is  of  information  overload. 

It  is  the  thesis  of  this  report  that  (I)  the  problem  is  frequently  not  information  overload  but 
lack  of  structure  and  organization  in  information  presentation,  and  that  (2)  appropriate  structuring 
can  lead  to  more  efficient  decision-making.  These  two  points  are  the  kernel  of  a  nascent 
information  clustering  hypothesis.  It  builds  on  the  concept  of  memory  chunks  by  linking  them  to 
chunks,  or  clusters,  of  information,  potentially  available  in  the  environment,  that  can  be  used  in 
expert  decision-making  and  problem-solving.  These  clusters  may  include  both  overt,  perceptual 
stimuli  available  to  all  decision-makers,  and  structural  or  configural  patterns  of  stimuli  used  by 
knowledgeable  experts.  The  first  part  of  the  hypothesis  is  nothing  more  than  a  restating  of  the 
well  known  observations,  died  above,  viz.,  that  experts  use  both  the  surface  and  deep  structural 
information  in  decision-making  and  problem-solving  and  that  they  have  a  large  amount  of 
competed  knowledge  in  their  area  of  expertise.  The  second  part  of  the  hypothesis  received  an 
initial  test  in  the  study  reported  here. 

The  experimental  task  chosen  was  simulated,  on-line,  college  course  scheduling,  which  has 
several  features  in  common  with  other,  real-world,  complex  decision  tasks.  To  simulate  an  event- 
driven  environment,  classes  could  fill  while  the  student  was  selecting  a  schedule.  When  a  planned 
course  was  filled,  the  subject  had  to  reassess  the  situation  and  find  a  new  course  that  fulfilled  the 
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other  requirements.  Elements  of  data  history  were  important  because  previous  semester's  records, 
program  requirements,  and  course  prerequisites  had  to  be  reconciled.  A  set  of  sometimes 
conflicting  goals  further  constrained  choices  (see  table  1).  Lastly,  of  course,  classes  could  not 
conflict  with  one  another. 


Table  1.  Goals,  Listed  in  Priority  Order 


1 .  Register  for  1 S  to  17  hours  (5  courses). 

2.  Try  to  fUlfill  requirements  and  prerequisites  for 
both  general  education  and  your  miyor- 

(Note  that  you  may  remain  an  industrial  engineering  major 
or  select  engineering  psychology.) 

3 .  Try  to  schedule  courses  to  allow  one  fUll  day  or  two  half  days  off. 
(One  full  day  is  preferable.  Assume  that  you  have  a  job  and 

will  otherwise  have  to  work  on  the  weekend.) 

4.  Try  to  get  the  best  instructors  possible.  (On  page  3  S  there  is  a 
list  of  instructor  ratings  from  a  student  survey.) 

5.  Try  to  avoid  8  a.m.  classes. 


The  scoring  scheme  operationalized  each  of  these  elements  as  values  associated  with 
accomplishing  the  goals  and  values  for  each  of  the  choices  (i.e.,  courses  and  instructors).  It  was 
predicted  that  scores  would  reflect  the  level  of  organization  in  the  display  schema,  with  better 
performance,  less  effort,  and  shorter  time-on-task  for  the  better  organized  displays. 
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Reverse  Blank 


METHOD 


SUBJECTS 

The  two  a  priori  requirements  for  subjects  were  that  they  be  recent  college  students  with  a 
minimum  of  eight  semesters  and  that  they  had  registered  for  college  classes  within  the  past  5 
years.  Thus,  they  could  be  considered  experts  at  the  putative  task.  There  were  10  women  and  26 
men  evenly  distributed  across  the  three  conditions  (designated  A,  B,  and  C).  They  had  last 
registered  for  college  courses  an  average  of  2.86  years  ago.  The  educational  level  of  the  sample 
ranged  from  bachelor  to  doctorate  degrees.  All  subjects  ranged  from  experienced  to  expert 
computer  users,  according  to  self-ratings.  The  subjects  did  nut  differ  significantly  on  any  relevant 
measures.  All  were  volunteers  and  were  not  paid  to  participate  in  this  experiment. 


APPARATUS  AND  MATERIALS 

Three  prototype  information  presentation  formats  were  developed  using  Supercard  on  a 
Macintosh  IDc  computer  with  a  19-inch  color  display.  Conditions  A  and  B  used  a  booklet-like 
format  with  only  a  single  page  visible  at  a  time.  The  page  numbers  in  the  Table  of  Contents 
functioned  as  buttons  to  provide  access  to  any  category  of  information.  Categories  are  listed  in 
table  2,  which  is  actually  the  table  of  contents  for  condition  B. 

In  condition  A,  all  information  in  the  course  schedule  and  prerequisites  sections  was  listed 
in  alphabetical  order.  Because  courses  were  not  listed  by  course  title  (only  course  number)  in  the 
requirements  and  student  record  sections,  this  format  required  a  search  information  retrieval 
strategy. 

In  condition  B,  course  schedule  and  prerequisites  were  listed  by  department  and 
sequentially  by  number  within  a  department.  This  partitioning  into  departmental  groupings  was 
analogous  to  the  functional  organization  employed  by  many  current  engineered  systems. 

Condition  C  used  a  format  that  supported  information  clustering  for  better  integration  of 
requirements,  course  availability,  student  record,  and  prerequisites  information.  It  used  a 
computer  registration  analogy  with  access  to  information  via  a  menu.  Multiple  movable  windows 
were  available  simultaneously.  These  were  scrollable  and  resizable  when  required  (e.g.,  course 
schedule,  registration  card,  and  any  other  card  with  more  than  about  10  lines  of  information). 
Information  clusters  were  determined  by  analyzing  the  steps  required  to  select  a  single  course. 
Pilot  testing  confirmed  the  clustei  contents.  An  example  of  a  single  cluster  included  requirements, 
course  prerequisites,  courses  already  taken,  and  times  of  potential  courses.  Prerequisites  and 
courses  already  taken  were  provided  within  the  windows  listing  course  schedules.  Requirements 
could  be  observed  simultaneously  with  other  windows. 
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Table  2,  Categories  of  Information  in  Table  Of  Contents:  Functional  Display  Format 


Table  of  Contents 

Goals . 

. ii 

Student  Record . 

. 1 

General  Requirements . 

. 2 

Industrial  Engineering  Requirements . 

. 3 

Engineering  Psychology  Requirements . 

. 4 

Prerequisites . 

. 7 

COURSES  SCHEDULED 

Chemistry . 

. 17 

Computer  Science . 

. 19 

English . 

. 19 

History . 

. 21 

Industrial  Engineering . 

. 21 

Mathematics . 

. 23 

Physics . 

. 27 

Political  Science . 

. 29 

Psychology . 

. 29 

Sociology . 

. 31 

Zoology . 

. 33 

MAPS  AND  TABLES 

Campus  Map . 

. 16 

Table  of  Instructor  Ratings . 

. 35 

Table  of  Class  Size  and  Seats  Remaining.... 

. 39 

DESIGN 

The  task  was  a  second -year  college  registration.  Subjects  were  given  goals,  student  records 
of  course  and  grade  history,  prerequisites,  a  campus  map  with  walking  times,  and  instructor 
ratings.  The  task  was  timed  and  courses  closed  dependent  on  the  elapsed  time.  The  experiment 
used  a  between  subjects  design  with  12  subjects  per  group.  Three  classes  of  dependent  measures 
were  used.  The  first  was  total  time>on>task  (T),  the  second  reflected  outcome  performance  (P), 
and  the  third  measured  processing  activity  (A). 

Performance  was  defined  as  the  summation  of  the  following  four  performance  measures:  the 
number  of  credits  successfully  registered  (PI),  the  sum  of  the  requirements  scores  for  all  courses 
registered  (P2),  the  sum  of  the  scheduling  difficulty  scores  for  all  courses  registered  (P3),  and  the 
average  preference  score  for  the  instructors  of  all  selected  courses  (P4).  Each  of  the  four  scores 
was  determined  a  priori  ard  was  reflected  in  the  goal  set  given  to  the  subjects. 
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The  processing  measures  captured  various  aspects  of  the  effort  that  subjects  put  into  the 
task.  These  included,  number  of  registration  attempts  (Al),  number  of  class  close-outs  (A2), 
number  of  windows  used  (A3),  and  number  of  times  the  subject  iterated  back  and  forth  between 
any  two  windows  (A4),  One  fiill  cycle  from  window  a  to  b  to  a  to  b  was  counted  as  one  iteration, 
These  measures  were  conibined  into  an  overall  efficiency  measure.  Efficiency,  E,  was  defined  as  a 
function  of  mean  overall  performance,  m(Pi),  mean  amount  of  processing,  m(Aj),  and  total  time- 
on-task  required  to  achieve  that  level  of  performance,  T: 

E-m(Pi)/m(Aj)  +  T. 


PROCEDURE 

The  experiment  took  place  in  a  small,  sound-damped,  experimental  room.  All  instructions 
were  presented  on  the  computer  screen.  Subjects  were  given  introductory  instruction  on  the 
manipulation  of  objects  used  in  the  program  and  on  the  task.  There  was  a  practice  task  for  each 
condition  that  duplicated  all  of  the  screens,  interactions,  and  information  types.  During  the 
practice  and  before  the  actual  experimental  trial,  subjects  were  invited  to  ask  questions,  however, 
questions  about  strategy  were  not  answered.  No  questions  were  answered  after  the  experimental 
trial  began.  At  the  end  of  the  experimental  task,  subjects  were  asked  to  complete  a  computerized 
questionnaire  and  ware  debriefed.  The  questionnure  was  designed  to  ascerttdn  recall  of  relevant 
information,  task  strategy,  computer  and  college  registration  expertise,  and  any  comments  about 
the  experiment. 
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RESULTS 


The  groups  did  not  differ  on  any  of  the  measures  of  computer  experience  or  college 
registration  experience.  Table  3  shows  means  and  standard  deviations  for  all  behavioral  measures 
(T,  P,  A,  and  E).  To  facilitate  comparisons  among  measures,  all  scores  were  transformed  into 
standard  scores  with  a  mean  of  SO  and  a  standard  deviation  of  10. 


Table  3,  Means  and  Standard  Deviations  for  All  Measures  for  All  Groups 


Alphabetical 

M  SD 

Functional 

M  SD 

Integrated 

M  SD 

Performance  Measures 

PI 

48.40 

11,77 

50.97 

6,36 

50.60 

11.62 

P2 

48.33 

12,20 

50.17 

8.87 

51.52 

9.23 

P3 

49.65 

10.07 

47.69 

11.35 

52.59 

8,73 

P4 

49.67 

11.46 

46.61 

8.60 

53.82 

9.29 

Process  Measures 

A1 

48.89 

5.43 

53.47 

15.04 

47.21 

5,76 

.A2 

51.01 

12,33 

52.15 

10,93 

46.44 

3.95 

A3 

55,78 

8.98 

54.21 

7.60 

40.02 

3.92 

A4 

56,65 

9.65 

53.84 

6.37 

39.51 

0.42 

Time-on-task 

49.24 

9.76 

52,54 

10.62 

48,21 

9.94 

Efficiency 

0.48 

0.07 

0.47 

0.09 

0.58 

0,08 

EFFICIENCY 

Major  interest  was  in  the  efficiency  of  decision-making  with  the  three  formats.  Subjects 
who  used  the  integrated  information  format  were  significantly  more  efficient  in  their  decision¬ 
making  than  were  those  using  either  of  the  other  two  formats,  £(2,33) «  7.29,  ]2<  005  (sec 
figure  1).  This  was  a  robust  effect,  with  h^  ■■  0.32.  In  a  post  hoc  test  for  trend,  the  quadratic 
trend  was  significant,  £(1,33)  “  4.32,  g  <  0,05. 
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GROUP 


Figure  1.  Efficient^  for  Each  of  the  Three  Display  Formats 
(performance  dMded  by  processing  effort  and  time) 


PROCESSING  MEASURES 

Detailed  examination  of  the  data  shows  the  contribution  of  performance,  processing,  and 
time  to  overall  efficiency.  The  process  measures  showed  two  patterns  of  behavior  (see  figure  2). 
For  ease  of  analysis  and  discussion,  these  shall  be  called  Ai,2  and  As, 4,  with  the  understanding  that 
the  two  components  of  each  compound  measure  displayed  the  same  pattern  of  results.  The  first 
compound  measure  was  composed  of  measures  Ai,  registration  attempts,  and  A:,  number  of 
close-outs.  These  are  both  indications  of  difficulties  with  the  task,  rather  than  the  display  format. 
There  were  no  significant  differences  among  the  conditions  on  this  compound  measure,  £(2,33)  - 
1.71,  n.s. 

The  second  pair  of  processing  measures;  A3,  number  of  windows  and  A4,  number  of 
iterations,  are  indications  of  information  usage  and  ease  of  access  for  each  display  format.  There 
were  significant  differences  among  the  groups  on  this  pair  of  measures,  £(2,33) «  43.04,  ]2<0.001. 
This  was  a  very  robust  effect  with  h^  -  0.72  and,  in  a  post  hoc  test  for  trend,  the  quadratic  trend 
was  significant,  E(l,33)"13,104,  p<0.005. 

The  iteration  measure  was  particularly  interesting.  In  conditions  A  and  B,  subjects  iterated 
between  members  of  information  clusters,  while  in  condition  C,  they  typically  positioned 
information  windows  so  that  all  members  of  relevant  clusters  were  visible  simultaneously.  Thus,  a 
high  number  of  iterations  is  indicative  of  an  unsupported  memoiy  load  requiring  subjects  to  return 
to  the  previous  window  to  re-access  information.  A  typical  p  ttem  for  the  subjects  in  condition  A 
was  to  iterate  between  the  schedule  page  and  virtually  every  other  page.  However,  there  were 
numerous  iterations  among  other  pages.  For  subjects  in  condition  C,  virtually  all  of  the  iterations 
were  between  the  schedule  page  and  one  of  the  pages  listing  course  schedules.  There  were  vetv 
few  iterations  for  subjects  in  condition  C,  and  these  were  different  for  each  subject.  Figure  3 
shows  typical  patterns  of  iterations  for  subjects  in  each  of  the  three  conditions. 
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Figure  2.  Mean  Processing  Score  on  Two  Compound  Measures  for  Each  Display  Format 
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CONDITION  C 


Figure  3.  Typical  Patterns  of  Iterations  for  Subjects  in  Each  of  the  Three  Conditions 
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TIME-ON-TASK 


While  means  were  not  significantly  different  among  groups,  variances  were  large,  and  time- 
on-task  did  contribute  to  individual  performance  differences,  Thus,  to  account  for  differences  in 
time  taken  by  individual  subjects,  performance  was  computed  per  unit  time. 


PERFORMANCE  MEASURES 

The  performance  measures  appeared  to  be  composed  of  two  compound  measures  that 
behaved  very  differently  (see  figure  4),  The  firat,  £1,2,  was  composed  of  the  more  concrete 
performance  measures;  Pi,  the  number  of  credits  successfully  registered  and  £2,  the  sum  of  the 
requirements  scores  for  all  courses  registered.  The  tasks  represented  by  these  measures  were 
essential  for  completion  of  the  course  schedule  and  did  not  reflect  differences  in  performance. 
They  replicated  minimal  or  baseline  performance.  Although  there  appears  to  be  a  slight  trend 
toward  better  performance  for  groups  B  and  C,  this  was  not  significant,  £<1.0. 

The  second  compound  performance  measure,  £3,4,  was  composed  of  more  difficult, 
evaluative,  and  integrative  tasks;  P3,  the  sum  of  the  scheduling  difficulty  scores  for  all  courses 
registered,  and  P4,  the  average  preference  score  for  the  instructors  of  all  selected  courses  (see 
figure  4).  These  measures  appeared  to  reflect  an  added  effort  to  perform  well,  when  possible. 
£3,4  showed  a  paradoxical  dip  in  performance  with  the  functional  display  organization,  B.  This 
quadratic  trend  was  marginally  significant,  £(1,33)  ■■  4.07,  p  ■  0.05. 


Figure  4.  Mean  Performance  on  Two  Compound  Measures  for  Each  Display  Format 
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To  complete  the  picture,  the  relationship  between  performance  and  process  measures  was 
analyzed.  This  relationship  can  best  be  understood  by  examining  figure  S.  As  can  be  seen,  there 
was  an  inverse  relationship  among  these  measures.  Processing  variables  were  moderately 
predictive  of  total  performance  score,  R* »  0.42. 


QROUP 


Figure  5.  The  Relationship  Between  Performance  and  Process  Measures  for  Each  Format 
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CONCLUSIONS  AND  RECOMMENDATIONS 


Overall,  the  results  indicate  the  importance  of  information  clusters  and  accessibility  for 
efficient  executive  decision-making.  More  detailed  examination  of  these  data  imply  a  relationship 
between  memory  load  and  processing  load  (effort)  for  information  usage  requirements.  The 
iteration  measure  is  especitdly  interesting  because  it  is  an  indication  of  whether  the  clusters 
provided  met  the  needs  of  the  decision-maker.  It  is  assumed  that,  where  the  user  iterated  between 
any  two  windows  several  times,  she  or  he  was  creating,  in  memory,  the  clusters  that  the  display 
did  not  provide. 

These  results  support  the  information  clustering  hypothesis.  The  three  versions  of  this  task 
had  the  same  performance  requirements  and  provided  the  same  information  to  the  subject,  but  ' 
group  C,  using  the  display  that  clustered  information  according  to  task  requirements,  was 
significantly  more  efficient  than  either  of  the  other  groups.  Interestingly,  there  was  no  significant 
difference  among  the  groups  on  the  gross  performance  measure,  or  on  the  easy  performance 
component,  Pi,3,  but  there  was  a  significant  difference  on  the  more  difficult  component,  P3,4. 

More  importantly,  the  trend  of  this  relationship  was  quadratic,  indicating  that  performance 
decreased  for  the  group  using  the  functional  partitioning  information  schema.  This  same  shape 
trend  was  found  for  processing  component  pUa  and  for  the  gross  efficiency  measure. 

These  three  trends,  taken  together,  suggest  that  a  functional  partitioning  of  data  can  actually 
be  detrimental  when  the  user  is  doing  an  integrative  task.  As  usual,  the  key  to  designing 
appropriate  display  schemas  is  to  understand  the  user's  task.  For  the  senior,  executive-level 
decision-maker,  the  task  is  frequently  integrative.  Providing  only  the  decomposed  functionality  is 
like  examining  the  results  of  a  factorial  study  without  considering  interaction  effects. 

These  results  suggest  that  a  display  that  organizes  information  according  to  function  can 
actually  hinder  efficient  performance  when  the  task  requires  integration  of  information.  They 
show  that  a  functional  display  organization  leads  to  longer  time  to  decide,  poorer  performance  on 
some  aspects  of  decision-making,  and  lower  efficiency,  as  compared  to  integrated  organization. 
This  result  is  only  reasonable  if  the  task  requires  integration  of  information  from  meny  functions. 
The  functional  display  organization  is  reasonable  if  the  task  requires  performing  the  frmction,  but 
not  if  it  requires  using  the  results  of  the  function  to  make  decisions. 

While  decision  aids  have  always  been  developed  by  analyzing  perceived  needs,  a  radically 
different  suggestion  is  to  design  information  management  decision  aids  the  way  new  walkways  are 
sometimes  planned.  That  is,  where  natural  paths  occur  because  of  repeated  use  by  pedestrians, 
constructed  (concrete,  macadam,  gravel,  etc.)  pathways  are  built.  These  are  not  planned  a  priori, 
but  develop  from  use.  Only  after  the  grass  has  been  worn,  are  constructed  paths  built.  In  the 
same  way,  the  interface  designer  might  track  the  way  users  employ  their  minimally  constrained 
systems  and  then  build  easy  access  pactis  among  frequently  used  items,  or  even  allow  the  user  to 
build  them.  The  naturally  occurring  pathways  worn  into  the  grass  are  indications  of  natural 
chunks  that  can  better  support  executive  decision-makitig. 
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